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Background: Medical students are required to retain vast amounts of medical knowledge on 
the path to becoming physicians. To address this challenge, multimedia Web-based learning 
resources have been developed to supplement traditional text-based materials. The Picmonic 18 
Learning System (PLS; Picmonic, Phoenix, AZ, USA) is a novel multimedia Web-based learn- 
ing platform that delivers audiovisual mnemonics designed to improve memory retention of 
medical sciences. 

Methods: A single-center, randomized, subject-blinded, controlled study was conducted to 
compare the PLS with traditional text-based material for retention of medical science topics. 
Subjects were randomly assigned to use two different types of study materials covering several 
diseases. Subjects randomly assigned to the PLS group were given audiovisual mnemonics 
along with text-based materials, whereas subjects in the control group were given the same 
text-based materials with key terms highlighted. The primary endpoints were the differences in 
performance on immediate, 1 week, and 1 month delayed free-recall and paired-matching tests. 
The secondary endpoints were the difference in performance on a 1 week delayed multiple- 
choice test and self-reported satisfaction with the study materials. Differences were calculated 
using unpaired two-tailed f-tests. 

Results: PLS group subjects demonstrated improvements of 65%, 161%, and 208% compared 
with control group subjects on free-recall tests conducted immediately, 1 week, and 1 month after 
study of materials, respectively. The results of performance on paired-matching tests showed 
an improvement of up to 331% for PLS group subjects. PLS group subjects also performed 
55% greater than control group subjects on a 1 week delayed multiple choice test requiring 
higher-order thinking. The differences in test performance between the PLS group subjects and 
the control group subjects were statistically significant (P<0.001), and the PLS group subjects 
reported higher overall satisfaction with the material. 

Conclusion: The data of this pilot site demonstrate marked improvements in the retention of 
disease topics when using the PLS compared with traditional text-based materials. The use of 
the PLS in medical education is supported. 

Keywords: medical education, e-learning, computer-assisted instruction, multimedia 
instruction 

Introduction 

Medical students are often overwhelmed by the volume of information they are required 
to learn, 12 and the problem of information overload in medical education has been 
a running theme for more than a century. 3 Abraham Flexner attempted to tackle this 
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problem in his landmark 1910 report, 4 and throughout the 
20th century, there were at least 24 additional major reports 
that called for further medical education reforms, many of 
which cited the challenges posed by increases in medical 
knowledge. 5 Despite the improvements that Flexner's recom- 
mendations brought to bear, some have argued that little has 
changed in the post-Flexnerian era 6 and that the problem has 
remained. 7 There may be several factors contributing to this, 
including the manner in which information is traditionally 
disseminated to medical students. 

Medical education evolved as an instructor-centered 
model during the last century. 2,7 Students were uninvolved 
in shaping their educational experiences, 1 and traditional 
teaching approaches emphasized passive learning in the 
form of faculty lectures and lacked interactivity, as seen with 
text-based materials. 8,9 

In recent years, computer-assisted instruction (CAI), the 
Internet, and Web-based learning (WBL) have helped alter 
the medical education landscape. CAI was first introduced 
to medical education in the early 1960s. 10 Initially, there 
was great excitement, and institutions such as Ohio State 
University, Massachusetts General Hospital, and the 
University of Illinois created their own CAI systems and 
then shared them with other schools as part of a consortium 
established by the Lister Hill Center of the National Library 
of Medicine. 11-13 Excitement gave way to skepticism, as 
barriers such as machine incompatibility, the prohibitive 
costs of computing, and a lack of access to computers pre- 
vented widespread adoption. 9 With the advent of the personal 
computer in the mid- 1 980s and the rapid rise of the Internet, 
which allowed for the delivery of CAI via WBL, 14 there has 
been a proliferation of WBL self-study resources. 

This has empowered students to take greater control of 
their learning experiences 15,16 to tackle the issue of infor- 
mation overload. Medical education has become more 
learner-centered. 17,18 Students are able to learn actively at 
their own pace and to take advantage of rich, interactive, 
multimedia resources such as question banks, online videos, 
multimedia textbooks, and much more. 19 

The PLS is a multimedia WBL platform that uses 
audiovisual mnemonics in combination with text-based 
materials, self-assessment modalities, and an interactive 
user interface to teach medical sciences. It is designed 
to improve memory retention and exam performance in 
medical sciences by leveraging the benefits of mnemonic 
instruction. Mnemonics have been shown to not only increase 
memory retention but also to improve higher-order thinking. 20 
Achieving this enhanced "learning with understanding" is a 



major focus of efforts of the new science of learning because 
it allows students to better apply their knowledge in diverse 
problems and arenas. 21 For further discussion of the ben- 
efits of mnemonic instruction please see Levin and Levin; 20 
Rummel, Levin and Woodward; 22 and Scruggs et al. 23 

Studies have demonstrated that multimedia WBL in 
medical education can increase efficiency 24 and learning 8 
when compared with traditional teaching approaches. More 
research, however, is required to explore whether multimedia 
WBL can improve memory retention, performance in tasks 
that require higher-order thinking, and student satisfaction 
through the use of audiovisual mnemonics in medical 
education. 

Material and methods 

Subjects 

Two hundred fifty-seven first-year osteopathic medical students 
at Midwestern University Arizona College of Osteopathic 
Medicine in Glendale were recruited to participate in the study 
from February until March 2013. Eighty-eight volunteered, 
one of whom later withdrew after experimental session 1 . This 
participant was not included in the statistical analysis. The 
study protocol was reviewed and approved by the Midwestern 
University Institutional Review Board, and informed consent 
was obtained from study subjects. 

PLS audiovisual mnemonic WBL platform 

The PLS concept was created and developed by two of the 
authors (RR and AY) to help medical students learn and mem- 
orize medical science materials in a more effective manner. 
At the core of the PLS (see Figure 1 and Supplementary 
materials) are audiovisual mnemonics; each one is intended 
to convey important attributes and information about a 
medical science concept or topic. The subject matter includes 
concepts/topics that are commonly tested in medical school, 
including those that frequently appear on the Comprehensive 
Osteopathic Medical Licensing Examination and the US 
Medical Licensing Examination. 

In the process of creating a Picmonic® Card, a concept/ 
topic of interest is selected, and then the most important 
attributes are identified and encoded into memorable char- 
acters, using the keyword mnemonic strategy. For a discus- 
sion of the process of encoding information into this format, 
please see Levin and Levin. 20 A linear narrative is created to 
connect the characters together, and this story is illustrated 
as a picture and recorded as an audio file. These files, along 
with text-based definitions for each of the attributes and a 
topic summary, are then inputted into the PLS. Students can 
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Figure I Screenshot of the Picmonic® Learning System. 

Notes: This example topic is rhinovirus, which is portrayed by the rhino characters with virus-like legs. The Picmonic® Card includes major attributes of the topic, which 
are listed in the left pane. Each attribute is incorporated into the picture in a creative and memorable manner and is associated with salient information. For example, one 
attribute of rhinovirus is that it is a picornavirus, which is represented by the pickle corn character. A story sequence portrayed by the picture then binds the attributes of 
the topic together. 



access the PLS and their library of Picmonic* Cards via a 
Web site portal, where they can perform self-assessment 
by ranking their mastery of concepts/topics and identifying 
areas of weakness. 

Experimental design 

Six disease topics were identified as the subject matter for 
this study: Sturge-Weber syndrome, neurofibromatosis 
type 1, Ewing's sarcoma, giant cell tumor, osteosarcoma, 
and tuberous sclerosis. To minimize contamination, disease 
topics to which the first-year medical students had not yet 
been exposed in their medical education curriculum were 
selected. 

Audiovisual mnemonics and text-based materials 
were developed for these disease topics and inputted into 
the PLS according to the processes discussed previously. 
Self-assessment modalities of the PLS were inactivated 
because experimental constraints, including limited time 
and single use of the PLS, precluded the subjects' ability 
to take advantage of the feature. A text-only version of 
the content was created by aggregating the same text that 
was inputted into the PLS, including the definitions and 
topic summaries. Key terms were highlighted to emphasize 
important information. The audiovisual mnemonic version 
of the content differed from the text-only version in that it 
contained a colored, audiovisual mnemonic representation 
of the text and interactive components, such as the ability 
to zoom in on a certain character/attribute in the picture and 
view the associated definition. It did not contain highlighted 
key terms, as in the text-only version. 



Two parallel desktop computer applications with input 
and output functionality were developed to deliver the 
experiment contents that included tests, a distraction task, 
and a satisfaction survey, in addition to the study materials. 
These applications differed only in their provision of either 
audiovisual mnemonic study materials via the embedded 
PLS (application A) or text-based only study materials 
(application B). The applications were loaded onto computers 
in a testing center, half with application A and half with 
application B. 

Using a computerized algorithm, study subjects were 
randomly assigned (1:1) into the PLS group (those using 
audiovisual mnemonics via the PLS; application A; n=46) 
and a control group (those using text-based materials of the 
same information; application B; n=42) and were directed 
to the appropriate computer terminal. 

Session I 

Pretest 

On initiation of the experiment, the subjects took a 1 0-minute 
baseline knowledge free-recall pretest before exposure to 
the study materials. In this pretest, subjects were given only 
the headings of the 6 disease topics (eg, "Sturge Weber 
syndrome") and instructed to list all medical science facts 
they knew about the topic. 

Study period 

Subjects were given a 5-minute tutorial, which allowed them 
to familiarize themselves with the type of study materials they 
would encounter. A 30-minute study period followed in which 
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detailed information corresponding to the 6 disease topics 
that were previously introduced in the pretest was provided, 
the PLS group was given access to the audiovisual mnemonic 
version of the information, and the control group was given 
access to the text-based version. One subject from the control 
group withdrew from the study at this point (n=41). 

Immediate tests 

A 3-minute scrambled-word distraction task, analogous to 
that described by Rummel, Levin, and Woodward, 22 was 
administered to reduce the effect of the subject's short-term 
memory on test performance. First, a 10-minute free-recall 
test was given that was identical in format to the baseline 
knowledge free-recall pretest, except that the disease topics 
were listed in a different order. Next, a 10-minute paired 
matching test was given in which subjects were instructed 
to match medical science terms with the appropriate disease 
topics. 

Satisfaction survey 

Subjects evaluated their satisfaction and engagement with the 
study material they were assigned to by answering 5-point 
Likert items. The survey contained a balance of both positive 
and negative questions to minimize potential acquiescence 
bias. 

Session 2 

One week delayed tests 

One week after the 30-minute study period during 
session 1, subjects returned to the testing center and were 
given 1 0 minute free-recall and paired-matching tests exactly 
as in session 1 (items were listed in a different order). In 
addition, subjects were given a 21 -minute multiple-choice 
test to evaluate their ability to integrate knowledge and 
perform tasks requiring higher-order thinking. 

Session 3 

One month delayed tests 

One month after the 30-minute study period during session 1 , 
subjects returned once again to the testing center. They were 
given 10-minute free-recall and paired-matching tests exactly 
as in sessions 1 and 2 (items were listed in a different order), 
and they were given a postexperiment exit survey to assess 
their general learning habits. 

Scoring procedure 

Scoring of the free-recall tests was based on the num- 
ber of medical science facts correctly recalled about the 



disease topics. A master listing of possible correct answers 
was created before testing, and the subject's answers were 
matched against this list. Subject's answers that were not 
found on the master list but were deemed to be correct 
were subsequently added to this list. After all amendments 
to the master list were made, the maximum possible score 
was 57. Scoring of the paired-matching tests was based on 
the number of correctly matched medical science terms with 
the appropriate disease topics, with a maximum possible 
score of 20. Scoring of the multiple choice test was based 
on the number of correct answers, with a maximum possible 
score of 16. Each item on the satisfaction survey was scored 
from 1 to 5 in accordance with the 5-point Likert scale. The 
highest score on a positive question was 1, and the highest 
score on a negative question was 5. 

Endpoints 

The primary endpoints were the differences in performance 
between the PLS group and control group subjects on 
immediate, 1 week, and 1 month delayed free-recall and 
paired matching tests. These were selected as the primary 
endpoints because these tests were delivered at all 3 sessions, 
and thus were the most reliable markers for assessing subject's 
memory retention. Secondary endpoints were the difference 
in performance between the PLS group and control group 
subjects on a 1 week delayed multiple choice test and self- 
reported assessments of satisfaction and engagement with 
the study material. 

Statistical analysis 

The mean test score and standard deviation were calculated 
for each test taken by subjects in the PLS and control groups. 
The mean test scores for the control group were subtracted 
from the mean test scores for the PLS group, and two-tailed 
unpaired Mests were conducted on these values according to 
a type 1 error probability of 0.01 . The same process was con- 
ducted for all 5-point Likert items in the satisfaction survey 

Results 

To ensure there were no significant differences between 
the PLS and control groups, an assessment of the subject's 
baseline knowledge of the 6 disease topics was obtained by 
administering a free-recall pretest. The results of the pretest 
scores were very similar, with the experimental group 
knowing a mean of 0.74 medical science facts and the control 
group knowing a mean of 0.78 medical science facts about 
the diseases. The difference in baseline knowledge between 
the two groups was not statistically significant. 
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At the delayed testing session, subjects were asked 
whether they had studied any of the disease topics outside 
of the designated 30-minute study period in session 1; none 
reported doing so. The subjects who had used the audiovisual 
mnemonic version of the study materials scored higher 
on every test administered throughout the 3 sessions (see 
Table 1). The percentage differences in scores on free-recall 
and paired matching tests also increased over time, with the 
greatest difference seen in session 3 on the 1 month delayed 
tests (see Table 2). The differences in mean scores between 
the experimental and control groups were all statistically 
significant at the level of P<0.001. 

The results of the satisfaction survey demonstrated that 
subjects using the audiovisual mnemonic version of the study 
materials delivered via the PLS were more satisfied and 
engaged with the content than those using text-based-only 
study materials (see Table 3). On average, PLS group subjects 
were more likely to report their study materials as an efficient 
use of their time (mean difference, 2.35; P<0.001). Further, 
they claimed a greater ability to maintain focus (mean dif- 
ference, 2. 19; P<0.001). The differences in mean values on 
the 5-point Likert items between the PLS and control groups 
were all statistically significant at the level of P<0.001. 

Discussion 

The results of this experiment indicate that compared with 
subjects who used the text-only version of the content, 
subjects who used the audiovisual mnemonic WBL version 
of the content retained medical science information about 
disease topics to a greater degree and for longer. They also 
outperformed their text-only counterparts on an application 
task (multiple choice test) that demanded higher-order think- 
ing skills than those called for in measures of memory requir- 
ing only rote memorization. Of great significance is the higher 
reported satisfaction and engagement with the audiovisual 
mnemonic WBL content. Subjects felt it allowed them to 



Table 1 Mean scores by test 


Test 


Control 


Picmonic® Learning 




group (n=4l), 


System group (n=46), 




mean (SD) 


mean (SD) 


Pretest 


0.78 (1.29) 


0.74 (1.20) 


Session 1 : free-recall 


19.37 (8.75) 


31.91 (8.74) 


Session 1: paired matching 


10.71 (4.23) 


15.96 (2.72) 


Session 2: free-recall 


7.98 (5.38) 


20.83 (8.23) 


Session 2: multiple choice 


7.80 (2.70) 


12.13 (2.30) 


Session 2: paired matching 


8.41 (3.31) 


12.85 (3.25) 


Session 3: free-recall 


7.32 (6.51) 


22.52 (8.69) 


Session 3: paired matching 


2.78 (1.72) 


1 1.98 (3.1 1) 



Abbreviation: SD, standard deviation. 



Table 2 Difference in 


mean scores 


between 


the Picmonic® 


Learning System and control groups 






Test 


Mean score 


t-test 


Mean score 




difference' 


score 


percentage 








difference 13 


Pretest 


-0.04 


—0. 1 5 


-5% 


Session 1 : free-recall 


12.55= 


L LO<- 

o.oo 


L CO/ 
CO A> 


Session 1 : paired matching 


5.25= 


6.80= 


49% 


Session 2: free-recall 


12.85= 


8 71 = 


1 6 1 % 


Session 2: multiple choice 


4.33= 


7.98= 


55% 


Session 2: paired matching 


6.15= 


6.29= 


53% 


Session 3: free-recall 


1 5.2 1 = 


9.30= 


208% 


Session 3: paired matching 


10.38= 


17.32= 


331% 



Notes: Experimental group mean-control group mean; b (experimental group 
mean-control group mean)/control group mean; 'statistically significant at the level 
P-C0.00I. 



remember complex information to a greater degree and that 
it was a more memorable way to present the information. 
Fostering excitement with medical education content is an 
important goal any educator should strive for and must be a 
focus of learner-centered curricula. These results support pre- 
vious research into the use of multimedia medical education 
programs 8 and the use of mnemonic strategies for improving 
memory retention and the application of knowledge. 22 

The audiovisual mnemonic WBL version of the content 
delivered via the PLS has various components that do not 
exist in the text-only version, including audiovisual mnemon- 
ics and interactive graphical simulation. Given that audiovi- 
sual mnemonics are at the core of this multimedia program, 
the greatest benefits in memory retention and applicability 
of knowledge can likely be attributed to this aspect of the 
PLS. However, further research is required to determine 
how much each aspect of the PLS contributes to improved 
memory retention and applicability of knowledge. 

The results of this study have several limitations. The 
study was conducted at only a single osteopathic medical 
school, so the findings must be established at other addi- 
tional institutions to make them more generalizable. 
Although subjects using the audiovisual mnemonic WBL 
version of the content applied their information superiorly 
on a 1 week delayed multiple choice test, this test was not 
delivered immediately after the 30-minute study period, nor 
was it delivered after 1 month, as the free-recall and paired 
matching tests were. Thus, the change over time in ability 
to apply knowledge was unable to be assessed. Further, 
subject's performance on standardized medical exams, 
such as the Comprehensive Osteopathic Medical Licensing 
Examination and US Medical Licensing Examination, 
was not assessed. These are tests in which applicability 
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Table 3 Satisfaction survey 


Survey questions 3 


Control group, 


PLS group, 


Mean score 




mean (SD) 


mean (SD) 


difference 


Using this educational resource, 1 was able to recall the information well 


2.44 (0.84) 


4.41 (0.62) 


l.97 b 


Studying with this resource is an efficient use of my study time 


2.41 (0.92) 


4.36 (0.74) 


1.94" 


The presentation of the material did not keep me focused 


4.15 (0.88) 


2.24 (1.18) 


-1.91" 


Using this resource, 1 was able to remember complex information well 


2.24 (0.99) 


4.02 (0.75) 


l.78 b 


The presentation of material was boring 


4.12 (1.05) 


1.72 (0.89) 


-2.40 b 


The given resource failed to present the information in a memorable fashion 


3.93 (0.98) 


1.74 (0.71) 


-2.l9 b 


1 had a satisfying study experience 


2.00 (0.92) 


4.35 (0.67) 


2.35" 


This is an inefficient way to study complex information 


3.66 (1.15) 


2.57 (1.33) 


-l.09 b 


1 would look forward to studying more with this type of resource 


1.85 (0.88) 


4.40 (0.81) 


2.55" 


Using the given resource, 1 was able to maintain focus 


1.93 (0.89) 


4.1 1 (0.83) 


2.19" 


This study resource was ineffective in helping me remember the information 


3.71 (1.01) 


1.69 (0.87) 


-2.02" 


presented 








1 feel like 1 am going to forget about many of these topics after today 


4.56 (0.59) 


2.76 (1.07) 


-l.8l b 


Notes: a AII questions are on the 5-point Likert rating scale; b statistically significant at the level P<0.00 1 . 



Abbreviations: SD, standard deviation; PLS, Picmonic® Learning System. 



of knowledge has real implications for medical students' 
careers. Traditional teaching approaches often include 
text-based materials and faculty-delivered lectures among 
other modalities; this experiment's text-only version of the 
content did not include these other modalities. Inclusion of 
lectures, for example, in addition to text-based materials 
would allow for a better comparison with broader tradi- 
tional teaching approaches, not just traditional text-based 
materials. 

Medical education curricula include many different 
types of content areas and learning objectives 25 that may 
demand varying instructional strategies. 26 More research 
is required to determine exactly what aspects of medical 
education should be taught using audiovisual mnemonic 
WBL versus other modalities, including traditional 
teaching approaches. Finally, this research examines 
knowledge/performance; however, as Cook 27 has empha- 
sized, this is only a substitute for the most important 
outcome, which is effect on physician performance and 
patient outcomes. 28 " 30 Further study should attempt to 
tackle such questions. 

Conclusion 

This single-center pilot study suggests that audiovisual mne- 
monic WBL provides marked improvement compared with 
traditional text-based study materials for teaching the main 
attributes of disease topics. Audiovisual mnemonics delivered 
via the PLS can play a role in continuing the learner-centered 
revolution in medical education and this study supports 
their use to supplement and enhance traditional text-based 
instruction. 
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Supplementary materials 

Video I Walkthrough of a Picmonic® sample card. Available from: https.7/www. 
youtube.com/wa tch?v=gJoU-uy9nkY . 
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